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This  paper  presents  a  framework  that  can  be  adapted  to  perform  automated  mapping/scheduling  and 
architecture  trades  for  embedded  computing  environments.  The  framework  is  based  on  a  learning  automata 
model  whose  adaptable  nature  makes  it  invaluable  in  tools  that  provide  end-to-end  solutions  for  embedded 
systems  development.  The  framework  has  been  incorporated  into  one  such  tool.  Systems  and  Applications 
Genesis  Environment  (SAGE).  The  paper  begins  with  an  introduction  to  learning  automata  and  the 
architecture  of  the  framework.  The  following  sections  discuss  how  algorithms  for  automated 
mapping/scheduling  and  architecture  trades  are  constructed  within  the  framework. 

2.0  Learning- Automata  based  Framework 

The  framework  is  based  on  a  variable  structure  stochastic  automaton  (VSSA).  The  VSSA  guarantees 
robust  behavior  in  the  absence  of  complete  knowledge  of  the  solution  space,  and  has  rigorous  mathematical 
properties  that  can  be  exploited  to  develop  efficient  algorithms.  It  can  be  viewed  as  a  stochastic  finite  state 
machine  with  a  set  of  actions  and  associated  probabilities  that  help  learn  the  nature  of  an  unknown 
environment.  For  every  random  action  that  is  chosen,  the  environment  that  needs  to  be  learned  provides  a 
response  that  is  stochastically  related  to  the  chosen  action.  The  iterative  process  of  choosing  random 
actions  and  recording  the  responses  is  continued  until  the  solution  space  is  satisfactorily  explored,  as 
depicted  in  Figure  1.  A  learning  automaton  is  usually  represented  as  a  quintuple  { O.a.p.A.G },  it’s 
mathematical  representation  is  defined  below.  The  VSSA  however  can  be  mathematically  simplified  such 
that  each  state  corresponds  to  a  distinct  action  and  hence  the  output  function  mapping  G  becomes  an 
identity  mapping.  Hence,  the  VSSA  becomes  a  triple  { a, (3, A }.  The  nature  of  the  response  “(3”,  determines 
if  the  VSSA  is  a  P-model,  Q-model  or  S-model  automaton  [1],  The  learning  algorithm  A,  drives  the 
reinforcement  scheme  that  controls  the  probability  vector  of  the  actions. 

Mathematical  representation: 

•  The  state  of  the  automaton  at  any  iteration  ‘n’,  denoted  by  <|)(n)  is  an  element  of  the  finite  set 

®=  { 4)i,tf»2,---,4>s} 

•  The  output  of  the  automaton  at  the  iteration  ‘n’,  denoted  by  a(n),  is  an  element  of  the  set 

a  =  {ai,a2,...,ar} 

•  The  input  to  the  automaton  at  the  iteration  ‘n’,  denoted  by  p(n),  is  an  element  of  the  set 

p_=  {pi,p2,...,pm} 

Org=  { (a,  b) } 

•  A,  is  the  updating  algorithm  or  the  reinforcement  scheme. 

•  The  output  function  G(.),  determines  the  output  of  the  automaton  at  any  iteration  ‘n’  in  terms  of 
the  state  at  that  iteration. 

a  (n)  =  G  [<(»  (n)] 

•  Reinforcement  Scheme: 

P  (n+1)  =  A  (  P(n),  a(n),  p(n) ) 

Where  P(n)  is  the  probability  vector  at  iteration  ‘n’. 

Our  framework  consists  of  a  set  of  independent  learning  automata.  Each  with  their  own  set  of 
actions  but  with  a  common  learning  algorithm.  The  response  from  the  environment  is  translated  to 
individual  responses  for  each  of  the  automata  based  on  different  heuristics.  The  framework  adapted  to  the 
automated  mapping  algorithm  is  depicted  in  Figure  2.  The  important  elements  in  the  framework  are  the 
environment,  actions  for  each  of  the  automata,  and  the  reinforcement  scheme  or  learning  algorithm. 

3.0  Automated  Mapping/Scheduling 

In  automated  mapping/scheduling,  the  objective  is  to  match  and  schedule  application  tasks  to  a 
heterogeneous  suite  of  machines  such  that  pre-defined  performance  criteria  such  as  size,  weight,  power, 
latency  and  bandwidth  are  optimally  satisfied.  The  learning  automata  framework  can  be  adapted  to  achieve 
this  objective.  A  pictorial  representation  of  this  schema  is  shown  in  Figure  2. 
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Environment:  Consider  a  Heterogeneous  Computing  (HC)  system  model.  It  consists  of  the  application 
represented  as  a  task  flow  graph  (TFG)  and  the  target  architecture  represented  as  a  processor  graph  (PG). 
Different  cost  metrics  can  be  defined  for  the  HC  system  model  [2], 

Automata  Construction:  Every  task  in  the  TFG  is  associated  with  an  automaton.  The  set  of  actions  for 
each  of  the  automata  is  the  set  of  processors  (nodes)  in  the  PG.  An  action  therefore  corresponds  to  mapping 
a  task  to  a  processor.  The  probability  equations  for  the  reinforcement  scheme  and  five  heuristics  which 
were  developed  to  drive  the  learning  algorithm  are  omitted  here  due  to  space  restrictions  and  can  be  found 
in  [2],  Results  of  the  mapping  algorithm  are  shown  in  figures  4  and  5.  The  performance  metric  in  these 
graphs  is  “Minimizing  the  total  execution  time”.  The  behavior  of  the  algorithm  is  plotted  for  varying 
weights  and  number  of  application  tasks.  In  Figure  4,  the  communication  complexity  is  assumed  to  be 
medium.  It  can  be  seen  that  as  weight  is  reduced  and  the  number  of  tasks  increased,  the  optimality  in  the 
chosen  performance  metric  is  lost.  The  graph  also  depicts  the  graceful  degradation  in  performance.  In 
Figure  5  the  communication  complexity  is  assumed  to  be  low  and  similar  observations  can  be  made. 

Key  Benefits:  The  automated  mapping  algorithm  within  the  proposed  framework  provides  several  benefits. 
The  primary  advantage  is  the  ability  to  define  multiple  cost  metrics  that  drive  the  optimality  of  the  system 
individually.  The  construction  of  the  algorithm  is  such  that  the  system  model  is  made  independent  of  the 
learning  automata  model,  making  it  universally  applicable  to  any  application  domain.  This  feature  allows 
the  mapping  scheme  to  be  incorporated  as  a  plug-in  into  a  tool  for  application  development  for  embedded 
systems  like  SAGE,  thereby  adding  value  to  the  tool. 

4.0  Architecture  Trades 

An  architecture  trade  requires  the  automatic  construction  of  a  system  that  minimally  conforms  to  the 
design  specifications  provided  by  the  user.  This  architecture  can  then  be  fine-tuned  to  match  the  exact 
system  requirements.  The  proposed  framework  can  be  used  to  develop  algorithms  that  aid  the  architecture 
trades  process.  Figure  3  depicts  the  adaptation  of  the  framework  to  this  trades  process. 

Environment:  The  environment  here  is  the  detailed  architecture  for  an  embedded  computing  system.  The 
system  is  analyzed  for  design  requirements  and  a  figure  of  merit  is  used  to  reflect  the  health  of  the 
architecture.  The  figure  of  merit  is  used  to  generate  a  response  that  is  fed  to  the  automata  model. 

Automata  Construction:  Here  the  user  typically  has  to  specify  the  component  types  that  define  his  target 
system.  An  automaton  is  then  associated  with  every  component  type  contained  in  the  target  system.  For 
instance,  if  the  system  architecture  consists  of  a  general  purpose  processor,  two  ASIC  chips,  a  memory 
module  and  a  network  bus,  then  the  automata  model  will  contain  an  automaton  each  for  the  processor,  each 
of  the  ASIC  chips,  the  memory  module  and  the  network  bus.  The  actions  for  the  automata  will  correspond 
to  the  different  classes  of  components  that  are  available  under  each  type.  For  example,  if  the  available 
processor  classes  are  Pentium  III,  FPGA,  and  PowerPC  603,  then  the  automaton  for  the  processor  will  have 
three  actions  corresponding  to  each  of  the  processors. 

When  each  automaton  selects  an  action  at  random,  the  resultant  system  forms  the  target  for  the 
application.  The  target  is  analyzed  for  design  requirements  and  the  response  is  used  to  drive  the  algorithm 
iteratively  until  an  optimal  solution  is  reached. 

Key  Benefits:  As  can  be  seen  from  the  construction,  the  learning  automata  framework  that  was  used  for 
automated  mapping/scheduling  is  easily  adapted  for  architecture  trades.  This  particular  feature  of  the 
framework  makes  it  invaluable  for  tools  that  provide  end-to-end  solutions.  Due  to  its  randomized  nature, 
the  proposed  methodology  allows  the  designer  to  better  explore  the  design  space. 

5.0  Conclusion 

In  conclusion,  we  propose  a  framework  that  can  be  adapted  to  perform  automated  mapping/scheduling  and 
architecture  trades  for  heterogeneous  systems.  The  framework  is  based  on  an  automata  model  that  provides 
critical  advantages  over  existing  systems  for  both  mapping  and  trades  study.  The  utility  of  this  framework 
demands  special  attention  in  the  context  of  incorporating  into  application  development  tools  that  provide 
complete  end-to-end  solutions.  This  capability  has  been  incorporated  into  SAGE  [3],  a  systems  integration 
framework  for  embedded  systems. 
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Figure  1 :  Learning  Automaton  Model 


Figure  2:  Framework  for  Automated  Mapping/Scheduling 


Figure  3:  Framework  for  Architecture  Trades 
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Figure  4:  Performance  plot  of  mapping  algorithm  for  medium  communication  complexity 
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Figre  5:  Performance  plot  of  mapping  algorithm  for  low  communication  complexity 


Learning  Automata 

Learns  the  unknown  nature  of  an  environment 

Variable  structure  stochastic  learning  automaton 
a  quintuple  {(p,a,|3,A,G}  where: 

-  cp(n),  state  of  automaton;  cp=  { cp , , . . .  ,cps } 

-  a(n),  output  of  automaton;  a={0Cj,...,ar} 

-  (3(n),  input  to  automaton;  (3={(31,...,Pm} 

-  A,  is  the  learning  algorithm; 

-  G[.],  is  the  output  function;  a(n)=G[cp(n)] 
n  indicates  the  iteration  number. 


Learning  Automaton  Schematic 


Action 


Response 


a(n) 


P(n) 


Output 


Automaton 

<(|),a,(3,A,G> 

<states,  actions,  input,  learning  algorithm,  output  function> 


Input 


Probability  Vector 

pin),  action  probability;  the  probability  that 
automaton  is  in  state  j  at  iteration  n. 

Reinforcement  scheme 

If  a(n)  =  cCj  and  for  j  <>  i;  (j=l  to  r) 

p/n+1)  =  p/n)  -  g  [p/n)]  when  (3(n)=  0. 

p/n+l)  =  Pj(n)  +  h[p/n)]  when  J3(n)=  1. 

In  order  to  preserve  probability  measure, 

£  p  (n)  =  1,  for  j  =  1  to  r. 


contd  ... 

If  a(n)  =  0Cj 

r 

Pt(n+1)  =  Pi(n )  +  L  g(p/n)) 

j=l>  joi 


Pi(n+1)  =  Pt(n)  -  I  h(p/n)) 

j=l.  joi 


g(. )  is  the  reward  function 
h(.)  is  the  penalty  function 


when  |3(n)  =  0 


when  (3(n)  =  1 


Schematic  of  Proposed  Automata  Model  for 

Mapping/Scheduling 


Machine 


asi(n) 


Output 


resp 


Psi(n) 


Automaton  for  task  Sj 
<asi,(3si,Asi> 

<machines,  environment  response,  learning  algorithm  > 


Input 


Model  Construction 


Every  task  st  associated  with  an  S-model  automaton 
(VSSA). 

VSSA  represented  as  {asi,  (3si,Asi),  since  r  =  s 

-  CCS1  is  set  of  actions  asi  =  m0,  m|5  m|M|l 

-  [3  is  input  to  the  automaton,  [3S|  e  [0,  1] 
closer  to  0  -  action  favorable  to  system; 
closer  to  1  -  action  unfavorable  to  system 

-  AS1  is  reinforcement  scheme 

pj n)  -  action  probability  vector 

-  probability  of  assigning  task  st  to  machine  m 


contd  ... 


•  Automata  model  for  Mapping/Scheduling 

-  S-model  VSSA  is  used 

-  Each  automaton  is  represented  as  a  tuple 

{asi,|3si,Asi} 

-  ocs'  =  nig,  ml5  m|Mh1 

-  Psi  €  [0,  1] 

(closer  to  0  -  favorable,  1  -  unfavorable) 

-  If  ck(n)  is  better  than  ck(n-1) 

Ekresp  =  0  else  Ekresp  =  1 

-  Translating  Ekresp  to  |3si  (n)  requires  two  steps 


Translating  Ekrespto  (3Si 


pso  psi  ps,s,i 


contd  ... 


•  Step  1 :  Translate  Ekresp  to  jisik  (n),  where 

-  |j,sik  (n)  -  input  to  automaton  s,  with  respect  to  cost 
metric  ck 

-  achieved  by  the  heuristics 

•  Step  2:  Achieved  be  means  of  Lagrange's 
multiplier 

|C|  |C| 

Psi  (n)  =  1 V  psik  (n),  i=1  to  |S|-1 ;  I  \  =  1 ,  \  >  0 

i=i  j=i 

where  Xk  is  the  weight  of  metric  ck 


Cost 


Low  Communication  Complexity,  Machines  =  5 


n5.74E+04+ 

5.23E+04  to  5.74E+04 

N"  4.72E+04  to  5.23E+04 
4.21E+04  to  4.72E+04 
■  3.70E+04  to  4.21E+04 
3.20E+04  to  3.70E+04 
2.69E+04  to  3.20E+04 
,  2.18E+04  to  2.69E+04 
1.67E+04  to  2.18E+04 
1.17E+04  to  1.67E+04 
-  6.58E+03  to  1.17E+04 
|  1.50E+03  to  6.58E+03 


Medium  Communication  Complexity,  Machines  =  5 


70000 


R6.24E+04+ 

5.69E+04  to  6.24E+04 
I  5.15E+04  to  5.69E+04 
I  4.60E+04  to  5.15E+04 
•  4.06E+04  to  4.60E+04 
3.51E+04  to  4.06E+04 
2.97E+04  to  3.51E+04 
.  2.42E+04  to  2.97E+04 
E88E+04  to  2.42E+04 
E33E+04  to  E88E+04 
,  7.85E+03  to  E33E+04 
|  2.40E+03  to  7.85E+03 


Weight 
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Schematic  of  Proposed  Automata  Model  for 

Architecture  Trades 


Machine 


eval 


asi(n) 


Output 


Psi(n) 


Automaton  for  component  Sj 
<asi,(3si,Asi> 

components,  performance  evaluation,  learning  algorithm  > 


Input 


Model  Construction 


•  Every  component  of  the  HW  system  si  associated  with 
a  P-model  automaton  (VSSA). 

•  VSSA  represented  as  {asi,  (3si,Asi),  since  r  =  s 

-  0CS1  is  set  of  component  types  asi  =  c0,  c,,  C|M|_j 

-  P  is  input  to  the  automaton,  |3si  =  0,  1 

0  -  performance  favorable  to  system;  1  -  unfavorable  to  system 

-  AS1  is  reinforcement  scheme 

•  pJ n)  -  action  probability  vector 

-  probability  of  choosing  component  st  from  component  type 


contd  ... 


•  Automata  model  for  Architecture  Trades 

-  P-model  VSSA  is  used 

-  Each  automaton  is  represented  as  a  tuple 

{asi,(3si,Asi} 

-  asi  =  c0,  c1 ,  . . . ,  C|M|_i 

-  psi  e  0, 1 

(0  -  favorable,  1  -  unfavorable) 

-  If  ck(n)  is  better  than  ck(n-1) 

Peval=  0  else  Peval=  1 


Conclusions 


Adaptive  Framework  for  Mapping  and 
Architecture  trades 

Automata  models  allow  optimization  of  multiple 
criteria 

Efficient  /  gracefully  degradable  solutions 

Framework  construction  suitable  for  tool 
integration 

-  Mapping  algorithm  integrated  with  SAGE™ 

Provides  a  basis  for  systems  design  from 
application  to  the  embedded  FIW 


